A Layered Locality Sensitive Hashing based Sequence Similarity Search Algorithm for Web Sessions

In this article we propose a Layered Locality Sensitive Hashing Algorithm to perform similarity search on the web log sequence data. Locality Sensitive Hashing has been found to be an efficient technique for the approximate nearest neighbor search over a large database, as it has sub-linear dependence on the data size even for high dimension. Mining the large web log data to provide customised services to the users is one such area where similar sessions are required to be extracted quickly. The variety of session lengths adds extra complexity to this problem. To tackle this dimension variability, the concept of layering in introduced in locality sensitive hashing and a recently proposed web page similarity measure Psim is used. The proposed method is referred to as Layered Locality Sensitive Hashing based Sequence Similarity Search Algorithm or LaLSA in short. The similarity at the session level is computed using a fast sequence alignment technique FOGSAA. LaLSA achieves an average time gain of 81.88% with 97.2% accurate result when compared to the exact algorithm, on NASA and ClarkNet web log datasets. Therefore, LaLSA is a time efficient solution to perform similarity search between variable length sequences, where the outputs are almost as good as the exact ones.